\subsection {Separator}

What is a separator?

Separator is a character that has a function to separate a sequence of semantic related character.  The function of a separator is increase readability.

A separator is a character with a very discriminative meta-feature towards other characters in a sequence.

One contribution is that we find the separator automatically. We require a few examples to find the separator. 
Instead of define separator beforehand (e.g. spaces and punctuations), we can detect the probability that a certain bigram permutes, i.e. P(permutes | ab), where a and b are two characters. Then one of this two character can be an indication of an separator. 

Approaches:
1) Find separator using separator to shift token and check with levenshtein fails for strings of the type SamurAraujo = AraujoSamur. It does not generalize to any other string.
2) Find most discriminative character based on the distance to other character. It may solve the previous problem.

\subsection {Character Transformation}
For our task is important to put a string a in the same format than a string b. Those transformations mainly involve replacing upper-case character by lower-case, accented character to non-accented, and general transformation of one character into another.

We do not cover here correction of spelling mistakes, normalization of orthographic variants, noun derivation, or lemmatization.

Think about permutation at character level (e.g. change C to c) and permutation at position level (e.g. change order of tokens).

\subsection {Ideas}
Think about a chain of transformation. Start applying pre-computing mapping rules (e.g. Corporation -> Corp.), then apply character specific transformations (e.g. upper-case to lower-case, accent to non-accent) and finish with position permutation. This seems to be a logical sequence of transformation for our setting. Check this for dates and different string types (e.g. company names, peoples name, citations)